ENCODER-DECODER MODELS

Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models (P18-3)

Challenge:	Recent language generative models are mostly trained on large-scale datasets, while in some real scenarios, the training datasets are often expensive and would be small-scale.
Approach:	They propose a novel duality fine-tuning method to capture more information from limited data and build connections between tasks.
Outcome:	The proposed method can capture more information from limited data, build connections between separate tasks, and is suitable for less-data constrained generation tasks.

Copied to clipboard

Satoru Katsumata, Yukio Matsumura, Hayahide Yamagishi, Mamoru Komachi

GLM: General Language Model Pretraining with Autoregressive Blank Infilling (2022.acl-long)

Challenge:	Encoder-decoder models employ words that are frequently used in the training corpus but may still include noisy words.
Approach:	They propose a method for selecting more suitable words for learning encoders by utilizing co-occurrence information.
Outcome:	The proposed method outperforms the baseline method in Japanese-to-English translation and grammatical error correction tasks with an F-measure of 1.48 points higher.

Copied to clipboard

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang

CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation (2025.findings-acl)

Challenge:	Existing pretraining frameworks do not perform well for all tasks of three main categories, such as natural language understanding (NLU), unconditional generation, and conditional generation.
Approach:	They propose a general language model based on autoregressive blank infilling to address this challenge.
Outcome:	The proposed model outperforms BERT, T5, and GPT on a wide range of tasks across NLU, conditional and unconditional generation tasks.

Copied to clipboard

Guofeng Cui, Pichao Wang, Yang Liu, Zemian Ke, Zhu Liu, Vimal Bhat

Machine Translation

Challenge:	Large language models (LLMs) have shown great potential in natural language processing tasks, but their application to machine translation remains challenging due to pretraining on predominantly English-centric datasets.
Approach:	They propose a method that combines reward scores with model confidence to improve model selection for fine-tuning.
Outcome:	The proposed method outperforms existing methods in translation accuracy and data efficiency.

GraphRAG-Rad: Concept-Aware Radiology Report Generation via Latent Visual-Semantic Retrieval (2026.eacl-srw)

Copied to clipboard

Faezeh Safari, Hang Dong, Zeyu Fu, Aline Villavicencio

Unseen Entity Handling in Complex Question Answering over Knowledge Base via Language Generation (2021.findings-emnlp)

Challenge:	Existing encoder-decoder models suffer from hallucinations, generating plausible but incorrect medical findings.
Approach:	They propose a novel architecture that integrates biomedical knowledge through a latent visual-semantic retrieval approach.
Outcome:	The proposed architecture achieves competitive performance with strong results across multiple metrics.

Copied to clipboard

Xin Huang, Jung-Jae Kim, Bowei Zou

Natural Language Interfaces

Challenge:	Existing methods for complex question answering are limited in the search space of all possible relation paths.
Approach:	They propose a method that directly generates an executable SPARQL query without simplification.
Outcome:	The proposed method significantly outperforms the previous methods and has higher interpretability and computational efficiency than the previous ones.

Probing Physical Reasoning with Counter-Commonsense Context (2023.acl-short)

Copied to clipboard

Kazushi Kondo, Saku Sugawara, Akiko Aizawa

Reasoning

Challenge:	Recent language models have succeeded in tasks that do not require contextual reasoning, such as size comparison and prediction of event frequency.
Approach:	They propose to use a dataset to measure physical commonsense to infer size relationships between objects in contexts that fit physical common sense.
Outcome:	The proposed dataset consists of contexts that fit physical commonsense and those that don't . results show that language models fail to use verbs and make incorrect judgments .

Stateful Memory-Augmented Transformers for Efficient Dialogue Modeling (2024.findings-eacl)

Copied to clipboard

Qingyang Wu, Zhou Yu

Improving Factuality in Clinical Abstractive Multi-Document Summarization by Guided Continued Pre-training (2024.naacl-short)

Challenge:	Existing Transformers models are computationally expensive for long context inputs.
Approach:	They propose a transformer that can interchange information between memory states and context . they evaluate the efficiency of their model on three dialogue datasets and two language datasets .
Outcome:	The proposed model is compatible with existing transformer models and can preserve dialogue history information.

Copied to clipboard

Ahmed Elhady, Khaled Elsayed, Eneko Agirre, Mikel Artetxe

Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)

Challenge:	Existing methods for enhancing the factual accuracy of abstractive summarization models are not effective in fact-critical domains like clinical document summarizing.
Approach:	They propose a guided continued pre-training stage for encoder-decoder models followed by supervised fine-tuning on summarization.
Outcome:	The proposed approach improves the quality and factuality of the summaries and achieves the best-published results on the clinical document summarization task.

Copied to clipboard

Hengyuan Zhang, Dawei Li, Shiping Yang, Yanran Li

Different Speech Translation Models Encode and Translate Speaker Gender Differently (2025.acl-short)

Challenge:	Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions.
Approach:	They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding.
Outcome:	The proposed method could generate more specific definitions compared with state-of-the-art models.

Copied to clipboard

Dennis Fucci, Marco Gaido, Matteo Negri, Luisa Bentivogli, Andre Martins, Giuseppe Attanasio

Multilinguality

Challenge:	Recent studies on interpreting the hidden states of speech models have shown their ability to capture speaker-specific features, including gender.
Approach:	They propose to use probing methods to assess gender encoding across ST models.
Outcome:	The proposed models capture speaker-specific features, including gender, while older models do not . low gender encoding capabilities result in systems’ tendency toward a masculine default, a translation bias that is more pronounced in newer architectures.

Visual Spatial Description: Controlled Spatial-Oriented Image-to-Text Generation (2022.emnlp-main)

Copied to clipboard

Yu Zhao, Jianguo Wei, ZhiChao Lin, Yueheng Sun, Meishan Zhang, Min Zhang

Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization (P19-1)

Challenge:	Image-to-text tasks such as captioning and controllable image descriptions have received extensive attention for decades.
Approach:	They propose a new perspective for image-to-text to generate spatial descriptions by combining two objects in an image.
Outcome:	The proposed model is awe-inspiring and human-like, and the proposed end-to-end architecture is the better choice for their integration.

Copied to clipboard

Sangwoo Cho, Logan Lebanoff, Hassan Foroosh, Fei Liu

Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation (P18-1)

Challenge:	Despite the empirical success of multi-document summarization, most datasets remain small and the cost of hiring hu-1 is prohibitive.
Approach:	They propose a novel method for extractive multi-document summarization that measures redundancy between a pair of sentences based on surface form and semantic information.
Outcome:	The proposed method outperforms baseline methods on benchmark datasets and is particularly useful for documents created by multiple authors containing redundant yet lexically diverse expressions.

Copied to clipboard

Tiancheng Zhao, Kyusong Lee, Maxine Eskenazi

Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models (2022.findings-acl)

Challenge:	Existing encoder-decoder dialog models cannot output interpretable actions as in traditional systems.
Approach:	They propose an unsupervised discrete sentence representation learning method that integrates with existing encoder-decoder dialog models for interpretable response generation.
Outcome:	The proposed model can be integrated with existing encoder-decoder dialog models and discover interpretable semantics via either auto encoding or context predicting.

Copied to clipboard

Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, Yinfei Yang

Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization (2025.naacl-long)

Challenge:	Sentence embeddings are useful for language processing tasks, but it is unclear how to produce them from encoder-decoder models.
Approach:	They investigate the effects of scaling up sentence encoders to 11B parameters on sentence embeddings from text-to-text transformers (T5) .
Outcome:	The proposed models outperform the previous best models on both SentEval and SentGLUE transfer tasks.

Copied to clipboard

Zilu Tang, Rajen Chatterjee, Sarthak Garg

A Discrete CVAE for Response Generation on Short-Text Conversation (D19-1)

Challenge:	Machine Translation (MT) systems based on fine-tuned large language models (LLMs) are at a higher risk of generating hallucinations, which can severely undermine user’s trust and safety.
Approach:	They propose a method that intrinsically learns to mitigate hallucinations during the model training phase.
Outcome:	The proposed method reduces hallucinations by 89% on an average across three unseen target languages while preserving translation quality.

Copied to clipboard

Jun Gao, Wei Bi, Xiaojiang Liu, Junhui Li, Guodong Zhou, Shuming Shi

Large Scale Multi-Lingual Multi-Modal Summarization Dataset (2023.eacl-main)

Challenge:	Neural conversation models are easy to generate bland and generic responses . however, their improvement of generating high-quality responses is still unsatisfactory .
Approach:	They propose to use a discrete latent variable with an explicit semantic meaning to improve the conditional variational autoencoder on short-text conversation.
Outcome:	The proposed model outperforms various kinds of generation models under automatic and human evaluations and generates more diverse and informative responses.

Copied to clipboard

Yash Verma, Anubhav Jangra, Raghvendra Verma, Sriparna Saha

Leveraging WordNet Paths for Neural Hypernym Prediction (2020.coling-main)

Challenge:	a large dataset of document-image pairs and annotated multi-modal summarization data is needed for multi-lingual modeling . encoder-decoder models represent information comprising multiple modalities.
Approach:	They propose to use a multi-lingual summarization dataset to analyze multi-modal summarizing using multi-linguistic annotated data.
Outcome:	The proposed dataset is the largest multi-lingual multi-modal summarization dataset for 13 languages and consists of cross-lingual summarizing data for 2 languages.

Copied to clipboard

Yejin Cho, Juan Diego Rodriguez, Yifan Gao, Katrin Erk

Challenge:	Existing work on lexical relations based on distributed representations has differed widely.
Approach:	They propose a model that generates taxonomy paths for hypernym prediction using WordNet sequences.
Outcome:	The hypo2path model outperforms the best model by 4.11 points in hit-at-one (H@1) The proposed model outpersforms previous models by a factor of 0.9.

RelU-Net: Syntax-aware Graph U-Net for Relational Triple Extraction (2022.emnlp-main)

Copied to clipboard

Yunqi Zhang, Yubo Chen, Yongfeng Huang

Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation (2021.findings-emnlp)

Challenge:	Existing methods focused on capturing semantic information but failed to incorporate syntactic structures of the sentence, which is proved to contain rich relational information.
Approach:	They propose a framework to capture syntactic information for relational triple extraction by contracting dependency tree into a core relational topology and eliminating redundant information with graph pooling operations.
Outcome:	The proposed framework incorporates syntactic information for relational triple extraction.

Copied to clipboard

An Yan, Zexue He, Xing Lu, Jiang Du, Eric Chang, Amilcare Gentili, Julian McAuley, Chun-Nan Hsu

Elaborative Simplification as Implicit Questions Under Discussion (2023.emnlp-main)

Challenge:	Radiology report generation aims at generating descriptive text from radiology images automatically.
Approach:	They propose a weakly supervised contrastive loss method that generates descriptive text from radiology images automatically.
Outcome:	The proposed method outperforms previous work on correctness and text generation metrics for two public benchmarks.

Copied to clipboard

Yating Wu, William Sheffield, Kyle Mahowald, Junyi Jessy Li

Focused Attention Improves Document-Grounded Generation (2021.naacl-main)

Challenge:	Automated text simplification is often thought of as a monolingual translation task . this view fails to account for elaborative simplification, where new information is added into the simplified text.
Approach:	They propose to view elaborative simplification through the lens of the Question Under Discussion framework . they propose to model 1.3K elongations accompanied by implicit QUDs to investigate what writers elaborate upon .
Outcome:	The proposed framework provides a robust way to investigate what writers elaborate upon, how they elaborate, and how elaborations fit into the discourse context.

Copied to clipboard

Shrimai Prabhumoye, Kazuma Hashimoto, Yingbo Zhou, Alan W Black, Ruslan Salakhutdinov

ClozEx: A Task toward Generation of English Cloze Explanation (2023.findings-emnlp)

Challenge:	Document grounded generation is the task of using the information provided in a document to improve text generation.
Approach:	They propose two new document grounded generation tasks that use information provided in a document to improve text generation.
Outcome:	The proposed models outperform existing methods on automated and human evaluation for closeness to reference and relevance to the document.

Copied to clipboard

Zizheng Zhang, Masato Mita, Mamoru Komachi

Explainability & Interpretability in NLP

Challenge:	Existing tasks and datasets specifically designed for generating language learner explanations for cloze questions are lacking . clozing questions are used to assess language proficiency and enhance language learning .
Approach:	They propose a task ClozEx to generate explanations for cloze questions in LA . they use a curated dataset of clozing questions paired with explanations .
Outcome:	The proposed task generates fluent explanations for cloze questions in English as a second language learners.

Learning Neural Templates for Text Generation (D18-1)

Copied to clipboard

Sam Wiseman, Stuart Shieber, Alexander Rush

An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text (2023.findings-acl)

Challenge:	Encoder-decoder models are uninterpretable and difficult to control in terms of content.
Approach:	They propose a neural generation system using a hidden semi-markov model which learns latent templates jointly with learning to generate.
Outcome:	The proposed model learns useful templates and achieves strong performance nearing that of encoder-decoder models.

Copied to clipboard

Yova Kementchedjhieva, Ilias Chalkidis

When and how to paraphrase for named entity recognition? (2023.acl-long)

Challenge:	Standard methods for multi-label text classification rely on encoder-only pre-trained models . encoder decoder models have proven more effective in other classification tasks .
Approach:	They compare four methods for multi-label classification based on encoder-only models . they use a pre-trained model for multilabel text classification .
Outcome:	The proposed methods outperform encoder-only methods on complex datasets and labeling schemes.

Copied to clipboard

Saket Sharma, Aviral Joshi, Yiyun Zhao, Namrata Mukhija, Hanoz Bhathena, Prateek Singh, Sashank Santhanam

NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models (2023.findings-emnlp)

Challenge:	Named entity recognition (NER) is a key component underpinning many industrial pipelines for a variety of downstream applications.
Approach:	They propose to use back translation to annotate entity spans in generations and propose a paraphraser with a larger dataset.
Outcome:	The proposed method improves NER performance across different datasets with gold annotations and paraphrasing strength.

Copied to clipboard

Jongwoo Ko, Seungjoon Park, Yujin Kim, Sumyeong Ahn, Du-Seong Chang, Euijai Ahn, Se-Young Yun

Zero-Shot Crosslingual Sentence Simplification (2020.emnlp-main)

Challenge:	Structured pruning methods have proven effective in reducing the model size and accelerating inference speed in various network architectures.
Approach:	They propose a framework that narrows the encoder and shortens the decoder networks of encoder-decoder models.
Outcome:	The proposed framework reduces the number of decoder layers and improves generation quality.

Copied to clipboard

Jonathan Mallinson, Rico Sennrich, Mirella Lapata

Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity (D18-1)

Challenge:	Recent approaches to simplification have shown promising results with encoder-decoder models trained on large amounts of parallel data which often only exists in English.
Approach:	They propose a model which transfers simplification knowledge from English to another language while generalizing across languages and tasks.
Outcome:	Empirical results show that the proposed model performs better than unsupervised and pivot-based methods.

Copied to clipboard

Xinnuo Xu, Ondřej Dušek, Ioannis Konstas, Verena Rieser

On Robustness of Finetuned Transformer-based NLP Models (2023.findings-emnlp)

Challenge:	Existing encoder-decoder models for open domain dialogue generate generic, uninformative, and non-coherent responses.
Approach:	They propose to introduce a measure of coherence as the GloVe embedding similarity between dialogue context and generated response to improve output diversity.
Outcome:	The proposed model improves on the OpenSubtitles corpus in terms of BLEU score and diversity metrics.

Copied to clipboard

Pavan Kalyan Reddy Neerudu, Subba Oota, Mounika Marreddy, Venkateswara Kagita, Manish Gupta

Robustness in NLP

Challenge:	Pretrained Transformer-based language models have been finetuned for a large number of tasks.
Approach:	They characterize changes between pretrained and finetuned models with CKA and STIR metrics.
Outcome:	The proposed models are more robust to perturbations than BERT and T5 on classification tasks and generation tasks.

Spherical Latent Spaces for Stable Variational Autoencoders (D18-1)

Copied to clipboard

Jiacheng Xu, Greg Durrett

Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding (2024.findings-acl)

Challenge:	Variational autoencoders use a multivariate Gaussian latent variable to capture latent structure in data.
Approach:	They propose a variational autoencoder which uses a latent distribution instead of Gaussian . they find that the variational posterior averts the KL collapse by a fixed hyperparameter .
Outcome:	The von Mises-Fisher distribution averts the KL collapse and gives better likelihoods than Gaussian models across a range of modeling conditions.

Copied to clipboard

Yuu Jinnai, Ukyo Honda, Tetsuro Morimura, Peinan Zhang

Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers (2023.emnlp-main)

Challenge:	Existing decoding algorithms to generate diverse outputs are based on beam search or random sampling, thus their output quality is capped by these underlying decoding methods.
Approach:	They propose to add a diversity penalty to MBR decoding and a clustering problem to create diversity-promoting decoding algorithms by enforcing diversity objectives.
Outcome:	The proposed method achieves a better trade-off than the diverse beam search and sampling algorithms overall.

Copied to clipboard

Hosein Mohebbi, Grzegorz Chrupała, Willem Zuidema, Afra Alishahi

Sparse Attention with Linear Units (2021.emnlp-main)

Challenge:	'context mixing' is a feature of Transformers that is used to build up representations of acoustic and linguistic structure in speech models.
Approach:	They propose to use a French spelling quirk to probe context mixing in speech models to find out how to translate spoken words into written equivalents.
Outcome:	The proposed model incorporates cues to identify correct transcription, whereas encoder-decoder models relegate task to decoder modules.

Copied to clipboard

Biao Zhang, Ivan Titov, Rico Sennrich

Machine Translation

Challenge:	Recent studies have suggested that sparse attention mechanisms can be made more interpretable by replacing the softmax activation with its sparser variants.
Approach:	They propose a method to replace softmax activation with a ReLU to achieve sparsity in attention by layer normalization with either a specialized initialization or an additional gating function.
Outcome:	The proposed model is easy to implement and more efficient than previously proposed sparse attention mechanisms.

Fine-Tuning Encoder-Decoder Models with Contrastive Learning for In-Context Distractor Generation (2025.findings-emnlp)

Copied to clipboard

Elaf Alhazmi, Quan Z. Sheng, Wei Emma Zhang, Mohammed I. Thanoon, Haojie Zhuang, Behnaz Soltani, Munazza Zaib

Bayelemabaga: Creating Resources for Bambara NLP (2025.naacl-long)

Challenge:	Distractors are used to generate plausible but incorrect options for fill-in-the-blank questions . research studies focus on fine-tuning pre-trained models with data augmentation techniques to generate distractors .
Approach:	They propose a model that trains the model to recognize essential semantic features necessary to generate distractors.
Outcome:	The proposed model outperforms existing models on two public datasets.

Copied to clipboard

Allahsera Auguste Tapo, Kevin Assogba, Christopher M Homan, M. Mustafa Rafique, Marcos Zampieri

Multilinguality

Challenge:	a lack of well-structured multilingual datasets remains a challenge for machine translation in under-resource languages.
Approach:	They propose to create a multilingual dataset for machine translation in the Bambara language, the vehicular language of Mali.
Outcome:	The proposed dataset is the most extensive curated multilingual dataset for machine translation in the Bambara language, the vehicular language of Mali.

Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing (2022.emnlp-main)

Copied to clipboard

Linlu Qiu, Peter Shaw, Panupong Pasupat, Tianze Shi, Jonathan Herzig, Emily Pitler, Fei Sha, Kristina Toutanova

Topic-Guided Self-Introduction Generation for Social Media Users (2023.findings-acl)

Challenge:	Pre-trained language models struggle on out-of-distribution compositional generalization . recent work shows considerable improvements on many NLP tasks from model scaling .
Approach:	They evaluate encoder-decoder models up to 11B parameters and decoder-only models up 540B parameters . they compare scaling curves for fine-tuning, prompt tuning, and in-context learning methods .
Outcome:	The proposed scaling methods improve compositional generalization on many tasks . fine-tuning generally has flat or negative scaling curves on out-of-distribution compositional . larger models are better at modeling the syntax of the output space, the study finds .

Copied to clipboard

Chunpu Xu, Jing Li, Piji Li, Min Yang

PIXAR: Auto-Regressive Language Modeling in Pixel Space (2024.findings-acl)

Challenge:	Existing studies on social media use tags to profile users, but we have found that sentence-level self-introductions are more natural and engaging.
Approach:	They propose a novel topic-guided encoder-decoder framework that uses a user's tweeting history to generate a short sentence outlining their personal interests.
Outcome:	The proposed framework outperforms existing encoder-decoder models on a large-scale Twitter dataset and shows that it is more natural and engaging than previous approaches.

Copied to clipboard

Yintao Tai, Xiyang Liao, Alessandro Suglia, Antonio Vergari

XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations (2023.acl-long)

Challenge:	Recent work shows the possibility of building open-vocabulary large language models that operate on pixel representations.
Approach:	They propose a pixel-based autoregressive LLM that performs generative tasks . they propose an adversarial pretraining stage that improves the readability and accuracy of PIXAR by 8.1 on LAMBADA and 8.5 on bAbI.
Outcome:	The proposed model performs free-form generative tasks while keeping the number of parameters on par with previous encoder-decoder models.

Copied to clipboard

Yusen Zhang, Jun Wang, Zhiguo Wang, Rui Zhang

Speech-enriched Memory for Inference-time Adaptation of ASR Models to Word Dictionaries (2023.emnlp-main)

Challenge:	Existing models for cross-lingual semantic parsing are not able to perform tasks on a wide range of datasets.
Approach:	They propose a benchmark for cross-lingual semantic parsing that uses 22 natural languages and 8 meaning representations to translate queries into MRs.
Outcome:	The proposed benchmarks cover 22 natural languages and 8 meaning representations on 164 domains and 5 tasks covering a wide range of multilingual language models.

Copied to clipboard

Ashish Mittal, Sunita Sarawagi, Preethi Jyothi, George Saon, Gakuto Kurata

Enhancing Goal-oriented Proactive Dialogue Systems via Consistency Reflection and Correction (2025.acl-long)

Challenge:	Existing contextual biasing techniques require additional parameterization . state-of-the-art ASR systems often fail to recognize named entities or critical rare words .
Approach:	They propose an algorithm that uses nearest-neighbor matching to predict ASR models . a list of rare entities is indexed in a memory and then stored the best possible match .
Outcome:	The proposed algorithm improves the prediction of state-of-the-art ASR models on rare words . it prevents spurious matches by restricting to word-level matches .

Copied to clipboard

Didi Zhang, Yaxin Fan, Peifeng Li, Qiaoming Zhu

NumHG: A Dataset for Number-Focused Headline Generation (2024.lrec-main)

Challenge:	Unlike traditional dialogue systems, goal-oriented proactive dialogue systems focus on achieving specific objectives by actively guiding and anticipating user needs.
Approach:	They propose a model-agnostic two-stage Consistency Reflection and Correction framework that allows the model to reflect on discrepancies between generated responses and dialogue contexts and suggest possible corrections.
Outcome:	The proposed framework significantly improves the consistency between generated responses and dialogue contexts on three datasets.

Copied to clipboard

Jian-Tao Huang, Chung-Chi Chen, Hen-Hsen Huang, Hsin-Hsi Chen

Flipping Knowledge Distillation: Leveraging Small Models’ Expertise to Enhance LLMs in Text Matching (2025.acl-long)

Challenge:	a lack of fine-grained annotations for accurate numeral generation in headlines is a major roadblock . a new dataset, the NumHG, provides over 27,000 annotated numeral-rich news articles for detailed investigation .
Approach:	They propose a dataset that provides annotated numerals for headline generation . they evaluate five well-performing headline-generation models using human evaluation .
Outcome:	The proposed dataset provides annotated numeral-rich news articles for detailed investigation.

Copied to clipboard

Mingzhe Li, Jing Xiang, Qishen Zhang, Kaiyang Wan, Xiuying Chen

On Leveraging Encoder-only Pre-trained Language Models for Effective Keyphrase Generation (2024.lrec-main)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable capabilities in acquiring diverse knowledge, making them highly effective across a wide range of tasks.
Approach:	They propose a flipped knowledge distillation paradigm where LLM learns from SLM . they propose to reinterpret LLMs as encoder-decoder models using LoRA .
Outcome:	The proposed model has been deployed in an online application environment and validated on financial and healthcare benchmarks and real-world applications.

Copied to clipboard

Di Wu, Wasi U. Ahmad, Kai-Wei Chang

Sequence-to-Sequence Spanish Pre-trained Language Models (2024.lrec-main)

Challenge:	a new study examines the use of encoder-only pre-trained language models in keyphrase generation (KPG) keyphrases are phrases that condense salient information of a document.
Approach:	They propose to use encoder-only pre-trained language models in keyphrase generation . they also examine optimal architectural decisions for employing encoder only PLMs in KPG .
Outcome:	The proposed model outperforms general-domain seq2seq models in keyphrase generation.

Copied to clipboard

Vladimir Araujo, Maria Mihaela Trusca, Rodrigo Tufiño, Marie-Francine Moens